Method and apparatus for obtaining three dimensional distance information stereo vision

ABSTRACT

A method of forming a three-dimensional stereo vision is disclosed in which, in order to prevent the erroneous detection of an object point in the conventional method using two image focusing lens systems, three image points of an object point formed by logically and/or physically selecting three image focusing lens systems and three image sensing surfaces from a multiplicity of image points on an image sensing surface by using the fact that a positional relation among three image points of the same object point is similar to the positional relation among the three image focusing lens systems, and the positional information of three selected image points on the image sensing surface is used for obtaining three-dimensional distance information of the object point.

BACKGROUND OF THE INVENTION

The present invention relates to a visual system, and more particularlyto a method of and an apparatus for forming a passive three-dimensionalstereo vision which is capable of extracting three-dimensional distanceinformation.

The above method and apparatus are applicable to various fieldsrequiring an accurate, high-speed passive visual system such as anautomobile (for the purpose of guiding the automobile when theautomobile is driven into a garage, is parked, or runs on a snarl-uproad or a highway, and for the purpose of facilitating the navigation ofthe automobile on an ordinary road), vehicles other than the automobile,a robot, factory automation, laboratory automation, office automation,building automation, home automation, and precise measurement.

In other words, the method and apparatus are used in operatorlessrepairs, an inspecting operation, an operatorless wagon, andoperatorless crane, operatorless construction, civil engineeringmachinery, the assembly of parts, measurement of the number of queuingpersons, a burglar-proof system, a disaster prevention system, ablindman guiding system, a speed detector, a distance detector, anobject detector, an automatic focusing mechanism for each of amicroscope, an enlarger, a projector, a copying machine, an optical discapparatus, an image pickup device, a camera and others, character/figurerecognition, the recognition of a number plate, a stereoscopic camera(for a still or moving object) and a game/leisure machine.

In a conventional method of forming a three-dimensional stereo vision,two brightness data obtained by two eyes are caused to correspond toeach other (that is, corresponding points of the two brightness data aredetermined) by pattern matching techniques. What is meant by patternmatching techniques is that the scale (namely, measure) of a coordinatespace is enlarged or contracted at each of the points in that portion ofthe coordinate space which has brightness distribution, so that thedifference between the brightness distribution in the coordinate spacewhose scale has been changed and reference brightness distributionbecomes minimum under some criterion.

Such pattern matching techniques encounter with two problems, the firstone of which is as follows. The number of coordinate points where thescale of the coordinate space is to be enlarged or contracted, is equalto the number of bright points (herein referred to as "bright lines")formed on a sensing surface, and the position of each of the coordinatepoints can be freely changed, provided that the configurational order ofthe bright lines is not changed. Accordingly, a vast number ofcombinations of scale transformation are basically allowed, and acombination capable of minimizing the difference between the brightnessdistribution obtained after scale transformation and the referencebrightness distribution, is selected from a multiplicity ofcombinations. Thus, it takes a lot of time to carry out pattern matchingby digital processing.

The second problem of the pattern matching techniques is as follows. Asmentioned above, the scale transformation is made so that theconfigurational order of the bright lines is not changed. In a casewhere the brightness distribution at an object system is such that afirst group of bright lines which is formed on a sensing surface by thefirst image focusing lens system, is different in the configurationalorder of bright lines from the second group of bright lines which isformed on the sensing surface by the second image focusing lens system,pattern patching is attended with a fundamental error. This error isattended for an object system in which the spacing between two objectpoints in the direction of depth is greater than the spacing between theobject points in a direction corresponding to the change of longitudinalazimuth angle. It is to be noted that, when the object points and theimage focusing lens systems are in the same plane surface, the directionfrom the image focusing lens systems towards the object points in theabove plane surface is herein referred to as "direction of depth", andan angle between the straight line connecting the image focusing lenssystems and the straight line connecting one of the image focusing lenssystems with one of the object points is herein referred to as a"horizontal azimuth angle". Incidentally, an angle between the aboveplane surface and the image sensing surface is herein referred to as the"vertical azimuth angle", which will be mentioned later.

In a case where an object system is not formed of independent objectpoints but has continuous spatial distribution, the continuousdistribution is converted into discrete distribution to make it possibleto use digital processing. Accordingly, this case also encounters theabove-mentioned two problems.

Further, even when a large number of brightness distribution data areused for increasing the accuracy of pattern matching, these two problemsare unavoidable, provided that the above-mentioned pattern matchingtechniques are used.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a method of and anapparatus for forming three-dimensional distance information which areable to solve the problems of the prior art and to form a stereo visionin a short time without producing the fundamental error based upon thespatial distribution object points within an object system.

In order to attain the above object, a method of and an apparatus forforming three-dimensional distance information in accordance with thepresent invention uses three or more image focusing lens systems. Inmore detail, in the above mentioned and apparatus, special attention ispaid to the fact that a special relation exists between the geometricalpositional relation among image points (namely, bright lines) of thesame object point which are formed on a sensing surface by three or moreimage focusing lens systems, and the geometrical positional relationsamong the image focusing lens systems themselves and between the sensingsurface and the image focusing lens systems, and bright lines of thesame object point are selected from a multiplicity of bright lines onthe sensing surface by using a special relation, to eliminate the twoproblems of the conventional pattern matching techniques.

An example of the above special relation is such that, when three ormore image focusing lens systems (for example, eyes) are placed in aplane, and the plane is parallel to an image sensing surface, imagepoints (namely, bright lines) of the same object point which are formedon the sensing surface by the image focusing lens systems, are spacedapart from one another so as to have a geometrical positional relationsimilar to the geometrical positional relation among the image focusinglens systems. In other words, a figure formed by straight lines whichconnects the image focusing lens systems is similar to a figure formedby straight lines which connects image points of the same object point.Accordingly, when those ones of the bright lines on the sensing surfacewhich are formed by each of the image focusing lens systems, can beknown, for example, when a plurality of sensing areas each correspondingto one of the image focusing lens systems are provided on imaginaryplanes bright lines which are formed by different image focusing lenssystems and have a geometrical positional relation similar to thegeometrical positional relation among the image focusing lens systems,can be extracted, as candidates for corresponding points, from amultiplicity of bright lines. (Incidentally, even in a case where asingle physical sensing surface is used, if the image focusing lenssystems form the image of an object system at different time moments, aplurality of sensing areas will be formed in effect.)

Those candidates for corresponding points which are extracted in theabove-mentioned manner, can indicate all of the object points, exceptingan object point whose image point is not formed on a sensing surface.Moreover, even in a case where the brightness distribution at the objectsystem is such that the configuration order of bright points which areformed on a sensing surface by one of the image focusing lens systems,is different from the configurational order of bright points which areformed on the sensing surface by another image focusing lens system, theabove-mentioned fundamental error is not generated in selecting thecandidates.

In some cases, however, a virtual object point which does not existactually, may be extracted as an object, since bright lines satisfyingthe above special relation can be found. The probability of extractingthe virtual object point can be greatly reduced by increasing the numberof image focusing lens systems, disposing image focusing lens systems ina plane at random, or disposing image focusing lens systems in differentplanes. This is because these countermeasures make conditions fordetermining corresponding points severe.

All the candidates for corresponding points thus obtained may be used asthe corresponding points. In fact, the candidates can be reduced on thebasis of brightness data. In general, when it is possible to detectbrightness data, color data (namely, data on the frequencycharacteristics of incident light), data on the phase of incident light,data on the plane of polarization of incident light, and thedifferential value of each of these data with respect to time or space,these data and their differentiated values can be used for selectingappropriate ones from the candidates. Now, explanation will be made on acase where brightness data is used for reducing the candidates, by wayof example. When brightness ratio among bright lines of the same objectpoint which are formed on the sensing surface by the image focusing lenssystems is previously known, only corresponding points having the abovebrightness ratio are extracted. The brightness of each bright line isdependent upon the direction dependence of the reflectivity of theobject point and the distance between the object point and the sensingsurface. The distance between the object point and the sensing surfacehas been known when the candidates for the corresponding points areextracted. Accordingly, only corresponding points based upon realisticdirection dependence of reflectivity are extracted. In a case where dataother than the brightness data is used, corresponding points can beextracted by a method corresponding to the data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram for explaining a method of forming athree-dimensional stereo vision through a conventional pattern matchingtechnique using two brightness data which are obtained by two imagefocusing lens systems.

FIG. 2 is a schematic diagram for explaining the generation of afundamental error in the method of forming a three-dimensional stereovision through the conventional pattern matching technique.

FIG. 3 is a schematic diagram for explaining the principle of a methodof forming a three-dimensional stereo vision in accordance with thepresent invention.

FIG. 4 is a block diagram showing an apparatus for carrying out thefirst embodiment of a method of forming a three-dimensional stereovision in accordance with the present invention.

FIG. 5 is a block diagram (partly pictorial and partly schematic)showing an apparatus for carrying out the second embodiment of a methodof forming a three-dimensional stereo vision in accordance with thepresent invention.

FIGS. 6 and 7 are schematic diagrams for explaining a method of reducingcandidates for corresponding points in accordance with the presentinvention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

In order to facilitate a understanding of the present invention, theprior art will first be explained, by reference to FIGS. 1 and 2. FIG. 1is a schematic diagram for explaining the pattern matching for twobrightness data which are obtained by two image focusing lens systems,and this diagram shows a case where an object system is formed of threeobject points a, b and c. In the prior art, a first image focusing lenssystem and a second image focusing lens system are used, and brightnessdistribution is formed on a sensing surface by each of these imagefocusing lens systems. Referring to FIG. 1, three points a, b and cwhich are spaced apart from one another, make up the object system, andthe brightness distribution which is formed on the sensing surface bythe first image focusing lens system, is the discrete brightnessdistribution made up of three bright lines a1, b1 and c1. Further, thebrightness distribution which is formed on the sensing surface by thesecond image focusing lens system is the discrete brightnessdistribution made up of three bright lines a2, b2 and c2. When thebright lines a1, b1, c1, a2, b2 and c2 are obtained, it is determined bysome method which of the bright lines obtained by one of the imagefocusing lens systems corresponds to each of the bright lines obtainedby the other image focusing lens system. Then, the distance between thesensing surface and the object system in the direction of depth isdetermined on the basis of the distance l between corresponding brightlines, and the horizontal azimuth angle is determined on the basis ofthe distance lg indicating the geometrical positional relation betweenone image focusing lens system and a bright line formed by this focusinglens system. Three-dimensional distance information with respect to theobject system is then obtained by using the above distance in thedirection of depth, the above horizontal azimuth angle, and otherfactors.

In the prior art, the bright lines obtained by the first image focusinglens system are forced to correspond to the bright lines obtained by thesecond image focusing lens system (that is, corresponding bright linesare determined) by some pattern matching technique. Fundamentally, whatis meant by pattern matching techniques is that the scale (namely,measure) of a coordinate space is enlarged or contracted at each of thepoints in that portion of the coordinate space which has a brightnessdistribution, so that the difference between the brightness distributionin the coordinate space whose scale has been changed and referencebrightness distribution becomes minimum under some criterion. In thecase shown in FIG. 1, the change of the scale corresponds to the changeof each of the distances between the bright lines a2 and b2 and betweenthe bright lines b2 and c2, that is, the change of the positions of thebright lines a2, b2 and c2 without being attended with a change in theconfiguration order of these bright lines. It is meant by patternmatching that by changing the positions of the bright lines a2, b2 andc2 in the above manner, the positional relation among the bright linesa2, b2 and c2 becomes equal to the positional relation among the brightlines a1, b1 and c1 (that is, the brightness distribution including thebright lines a2, b2 and c2 becomes entirely equal to the brightnessdistribution including the bright lines a1, b1 and c1).

FIG. 2 is a schematic diagram for explaining the generation of thepreviously-mentioned fundamental error in a case where an object systemincludes two object points so that the spacing between the object pointsin the direction of depth is greater than the spacing between the objectpoints in the direction corresponding to the change of horizontalazimuth angle. Referring to FIG. 2, the object system includes an objectpoint a', instead of the object point a of FIG. 1, and the spacingbetween the points a' and b in the direction of depth is greater thanthe spacing between the points a' and b in the direction correspondingto the change of horizontal azimuth angle, to such an extent that thepoint a' lies between two straight lines which connect the point b withthe first and second image focusing lens systems. As shown in FIG. 2,the brightness distribution which is formed on the sensing surface bythe first image focusing lens system, is different in the configurationorder of bright lines from the brightness distribution which is formedon the sensing surface by the second image focusing lens system. Thatis, the bright lines formed by the first image focusing lens system arearranged in the order of c1, a'1 and b1. While, the bright lines formedby the second image focusing lens system are arranged in the order ofc2, b2 and a'2. When pattern matching is performed for these brightnessdistribution data, there is a strong probability that the bright linea'1 is forced to correspond to the bright line b2 and the bright line b1is forced to correspond to the bright line a'2. As a result, the trueobject points a' and b are not detected from the brightness distributiondata on the sensing surface, but the object system is judged to includethe false object points m and n. That is, a fundamental error isgenerated.

Now, a case where pattern matching is performed for brightnessdistribution data obtained by three image focusing lens systems whichare disposed on a straight line parallel to a sensing surface, byutilizing the principle mentioned in "SUMMARY OF THE INVENTION", will bebriefly explained by reference to FIG. 3.

FIG. 3 shows a case where the third image focusing lens system is addedto the first and second image focusing lens systems of FIG. 2. In FIG.3, the first brightness distribution, which is formed on the sensingsurface by the first focusing system, is disposed in a first imaginaryplane which is spaced apart from a second imaginary plane in which thereis disposed the second brightness distribution, which is formed on thesensing surface by the second focusing system, by the distance l₁₂between the first and second focusing systems; and the third brightnessdistribution, which is formed on the sensing surface by the thirdfocusing system, is disposed in a third imaginary plane which is spacedapart from the second imaginary plane by the distance l₂₃ between thesecond and third focusing systems.

The above arrangement of the first, second and third brightnessdistribution in respective first, second and third imaginary planes canclearly show a special relation which exists between the geometricalpositional relation among bright lines of the same object point whichare formed by the focusing systems, and the geometrical positionalrelations among the focusing systems themselves and between the sensingsurface and the focusing systems. In the case of FIG. 3, the specialrelation is that a ratio of the distance between the first and secondones of three bright lines of the same object point to the distancebetween the second and third bright lines is equal to a ratio of thedistance between the first and second focusing systems to the distancebetween the second and third focusing systems. Accordingly, when thefirst brightness distribution formed by the first focusing system, thesecond brightness distribution formed by the second focusing system andthe third brightness distribution formed by the third focusing systemare arranged so that the ratio of the distance between the firstbrightness distribution and the second brightness distribution to thedistance between the second brightness distribution and the thirdbrightness distribution is equal to the ratio of the distance l₁₂ andthe distance l₂₃, three bright lines caused by the same object point lieon a straight line.

Conversely, when a group of bright lines lying on the same straight lineis found, candidates for corresponding points can be obtained. In FIG.3, straight lines each indicating corresponding bright lines areexpressed by broken lines A, A', B, C and P. One of the above straightlines can be determined in the following manner. That is, when a brightline in the brightness distribution formed by a focusing system (forexample, the second focusing system), that is, a bright line in thesecond brightness distribution is specified, and a bright line ispresent at each of two positions one of which is spaced apart from thepoint on the first brightness distribution corresponding to thespecified bright line in the leftward direction by a given distance, andthe other of which is spaced apart from the point on the thirdbrightness distribution corresponding to the specified bright line inthe rightward direction by the distance equal to the product of thegiven distance and a ratio l₂₃ /l₁₂, the above-mentioned straight linecan be obtained by connecting these positions. The extraction of theabove positions in the first and third brightness distribution capableof satisfying the condition with respect to the ratio l₂₃ /l₁₂ and theAND operation for two pieces of brightness information at the abovepositions can be readily and rapidly performed by hardware or software.

The remaining straight lines are determined in such a manner that eachof the bright lines in the second brightness distribution other than thespecified bright line is selected, and the presence of a bright line isconfirmed at those positions on the first and third brightnessdistribution which can satisfy the above-mentioned condition withrespect to the ratio l₂₃ /l₁₂ and another condition that the distancebetween the selected bright line and the above position on the firstbrightness distribution is different from the distance between thespecified bright line and that position on the first brightnessdistribution which has a bright line in the preceding processing.

In this case, a vast number of positioning operations are performed onthe first, second and third brightness distribution, and the number ofpositioning operations to be performed is substantially equal to theproduct of the number of bright lines in the second brightnessdistribution (that is, the number of object points in a plane which isdefined by the focusing systems and bright lines on the actual sensingsurface), the number of bright lines in the first or third brightnessdistribution which are to be combined with the bright lines in thesecond brightness distribution to determine the distance between theactual sensing surface and each object point in the direction of depth,and the number of bright lines which can be found on the actual sensingsurface along a direction corresponding to the change of verticalazimuth angle (that is, a direction parallel to a straight line at whicha plane perpendicular to a straight line connecting the first, secondand third focusing systems intersects with the actual sensing surface).

Now, let us consider an extreme case where the number of bright lines ineach of the first, second and third brightness distribution is equal tothe spatial resolving power (namely, resolution). When it is assumedthat the resolution is equal in all directions, the number ofpositioning operations is equal to the cube of the resolution at most.

In a case where the resolution is equal to 500, the number ofpositioning operations is 125×10⁶. If the positioning operations areperformed by serial, digital processing which uses a clock signal of 1MHz, a processing time proportional to 125 sec will be necessary. If aclock signal of 10 MHz is used, a processing time proportional to 12.5sec will be required.

In fact, the above positioning operations can be carried out by parallelprocessing, and thus the processing time can be greatly reduced. Forexample, brightness distribution data corresponding to those straightlines in an actual sensing surface which are parallel to the straightline connecting the first, second and third focusing systems and areequal in number to the resolution, can be processed at once by parallelprocessing. Accordingly, the frequency of processing becomes apparentlyequal to 10⁴. Thus, if the serial, digital processing using a clocksignal of 1 MHz is carried out, a processing time proportional to 0.25sec will be necessary. If the clock signal of 10 MHz is used, aprocessing time proportional to 0.025 sec. will be required. Further,the positioning operations for one of the second brightness distributionand the first or third brightness distribution can be performed inparallel when a bright line in the other brightness distribution isspecified. Thus, the frequency of processing becomes apparently equal to500. If the serial, digital processing using the clock signal of 1 MHzis used for the above processing, a processing time proportional to 0.5msec. will be necessary. If the clock signal of 10 MHz is used, aprocessing time proportional to 0.05 msec. will be required. Thepipeline method may be applied to the above 500 processing.

Further, all the possible combinations of positions in the secondbrightness distribution and positions in the first or third brightnessdistributions are fundamentally known. Accordingly, all processing canbe carried out in parallel. In this case, the processing is apparentlycarried out only once. Accordingly, if the serial, digital processingusing the clock signal of 1 MHz is utilized, the processing time will bein the order of 1 μs. If the clock signal of 10 MHz is used, theprocessing time will be in the order of 0.1 μs.

In a case where an object system is investigated in a limited range ofeach of the distance in the direction of depth, the horizontal azimuthangle and the vertical azimuth angle, for example, only 10% of each ofthe second brightness distribution, the first or third brightnessdistribution and the whole range of vertical azimuth angle, the numberof positioning operations is as small as 125×10³ even when the parallelprocessing is not used. Accordingly, if the serial, digital processingusing the clock signal of 1 MHz is utilized, the processing time will beabout 0.125 sec. If the clock signal of 10 MHz is used, the processingtime will be about 0.0125 sec.

Further, let us consider a case where the brightness distributionindicating edges in an object system can be used instead of thebrightness distribution indicating object points. When bright lines inthe second brightness distribution which indicate edges are 20% ofbright lines in the same brightness distribution which indicate objectpoints, bright lines in the first or third brightness distribution whichindicate edges are 20% of bright lines in the same brightnessdistribution which indicate object points, and those bright linesarranged on the actual sensing surface along a direction correspondingto the change of vertical azimuth angle which indicate edges, are 20% ofbright lines which are arranged along the above direction and indicateobject points, the number of positioning operations is equal to 10⁶.Accordingly, if the serial, digital processing using the clock signal of1 MHz is utilized, the processing time will be about 1 sec. If the clocksignal of 10 MHz is used, the processing time will be about 0.1 sec.

The straight lines for extracting candidates for corresponding points,that is, the broken lines A, A', B, C and P can be determined in theabove-mentioned processing time. As can be seen from FIG. 3, the brokenlines A, A', B and C correspond to actual object points a, a', b and c,respectively, but an object point p corresponding to the broken line Pdoes not exist actually. The appearance of such a virtual object pointis caused by the fact that the bright lines a1, a'2 and b3 are formed onthe sensing surface accidentally so that the ratio of the distancebetween the bright lines a1 and a'2 to the distance between the brightlines a'2 and b3 is equal to the ratio of the distance between the firstand second focusing systems to the distance between the second and thirdfocusing systems.

It can be known from FIG. 3 that the possibility of appearance of thevirtual object point is greatly reduced by additionally disposing afourth focusing system at a given position. If the fourth focusingsystem is disposed on the straight line connecting the first, second andthird focusing systems, the reduction of the above possibility will bereadily seen from FIG. 3.

Further, it can be supposed that the possibility of appearance of thevirtual object point is reduced by changing the geometrical arrangementof the first, second and third focusing systems, instead of adding thefourth focusing system to these focusing systems. It can be readily seenfrom FIG. 3 that the above possibility will be reduced, if the first,second and third focusing systems are moved on the straight lineconnecting these focusing systems so that the ratio of the distancebetween the first and second focusing systems to the distance betweenthe second and third focusing systems differs from the original ratiol₁₂ /l₂₃.

An extreme case where a continuous straight line is used as an objectsystem, will be explained below, by reference to FIG. 6. FIG. 6 shows acase where not only a line segment a c which is an object system, butalso first, second and third focusing systems are disposed in the sameplane. In this case, the image of every point in a common area of threetriangular regions each bounded by two straight lines connecting one ofthe first, second and third focusing systems with both ends of the linesegment a c and their extensions, can be formed on a sensing surface byeach of these focusing systems. Accordingly, it seems as if an objectsystem were present throughout the above common area, (namely, hatchedarea). In a case where an object system has a planar shape, a commonportion of three conical regions each bounded by a curved surfaceconnecting one of the first, second and third focusing systems with theperiphery of the object system and its extension, corresponds to thecommon area.

The hatched area of FIG. 6 can be eliminated by placing the thirdfocusing system at a position deviating from the plane, in which thefirst and second focusing systems and the line segment a c lie, as shownin FIG. 7. In FIG. 7, reference symbol 3' designates the new position ofthe third focusing system. In this case, a triangular region bounded bytwo straight lines connecting the third focusing system with both endsof the line segment a c and their extensions intersects with the aboveplane only at the line segment a c. Accordingly, the hatched area isextinguished.

Further, it can be readily known that the hatched portion can be madesmall by additionally disposing the fourth focusing system in the planewhich contains the line segment a c and the first, second and thirdfocusing systems, instead of deviating the third focusing system fromthe plane.

Also, in a case where an object system has a planar shape, the hatchedportion can be reduced by changing the geometrical arrangement of thefocusing systems or by increasing the number of focusing systems.Further, when the planar shape is converted into a linear shape bypreprocessing, such as edge extraction, or by decreasing the size of theplate, the portion can be made extremely small.

The broken line P which appears in FIG. 3 and corresponds to the virtualobject point p, can be removed by utilizing the brightness informationof bright lines. Now, let us consider a case where image points of thesame object point formed by three focusing systems are equal inbrightness to one another, as shown in FIG. 3. When at least one ofthree bright lines extracted by a straight line is different inbrightness from the remaining bright lines, this straight line iseliminated. Bright lines extracted by each of the broken lines A, A', Band C have the same brightness, but two bright lines a1 and a'2 of threebright lines a1, a'2 and b3 extracted by the broken line P are differentin brightness from the remaining bright line b3. Accordingly, the brokenline P is eliminated.

The processing for eliminating an unwanted straight line can be carriedout in a short time by software or hardware. Further, this processingand the above-mentioned positioning processing can be combined so as tobe simultaneously performed.

Next, let us consider a case where image points (namely, bright lines)of the same object point formed by a plurality of focusing systems donot have the same brightness but the reflectivity of an object point (orthe radiant intensity of a luminous object point) is equal in alldirections. The brightness of a bright line on the sensing surface isinversely proportional to the square of the distance between the objectpoint and the bright line. Accordingly, when brightness values eachobtained by multiplying the brightness of one of bright lines extractedby a straight line by the square of the corresponding distance, are notequal, the straight line is eliminated.

In most cases, the above-mentioned conditions with respect to brightnessare satisfied. Even in a case where the reflectivity of an object pointvaries with direction, if the extent of variations in reflectivity ispreviously known, the brightness of each of bright points extracted by astraight line will be corrected by the above information, and thus itwill be judged on the basis of the corrected brightness values of thebright lines whether the straight line is to be eliminated or not. Evenin a case where the distance correction or the correction for variationsof the reflectivity of an object point with direction is necessary, anunwanted straight line can be readily eliminated in a short time bysoftware or hardware, as in a case where bright points of the sameobject point formed by a plurality of focusing systems have the samebrightness.

It is to be noted that the prior art using a pattern matching techniquedetermines corresponding points on the assumption that bright lines onthe sensing surface are all equal in brightness.

After corresponding points have been determined in the above-mentionedmanner, both the information on the positions of at least two of threebright lines indicated by the corresponding points, on the sensingsurface, and the information on the geometrical positional relationbetween the focusing systems and the sensing surface are used forobtaining two-dimensional distance information of an object point, thatis, the distance between the object point and the sensing surface in thedirection of depth and one or two horizontal azimuth angles. Thetwo-dimensional distance information thus obtained is combined with thevertical azimuth angle information which is obtained independently ofthe above two-dimensional distance information, to obtainthree-dimensional distance information of the object point.

The processing for obtaining each of the two-dimensional distanceinformation and the three-dimensional distance information is aprocessing based upon simple abgebraic equations, and can be readilyperformed in a short time. The whole quantity of processing necessaryfor obtaining three-dimensional distance information of an object systemis proportional to the number of object points included in the objectsystem.

Now, explanation will be made of a first embodiment of a method offorming a three-dimensional stereo vision in accordance with the presentinvention, by reference to FIG. 4.

The first embodiment is carried out by an apparatus which, as shown inFIG. 4, includes image focusing systems 11 to 13 each formed of a lensor mirror, image sensing surfaces 21 to 23, a detected informationpreprocessing system 30, a detected information storage system 40, acandidate-for-corresponding-point decision system 50, afalse-corresponding-point removing system 60, a three-dimensionaldistance information calculation system 70, and a configuration changesystem 80 for changing the configuration of the image focusing systems11 to 13.

The image focusing systems 11 to 13 are not always required to lie on astraight line parallel to the sensing surfaces 21 to 23, but thepositions of the image focusing systems can be changed by theconfiguration change system 80 so that a plane containing the imagefocusing systems 11 to 13 makes a desired angle with the image sensingsurfaces 21 to 23.

The images of an object system which are formed on the sensing surfaces21 to 23 by the focusing systems 11 to 13, are sent to the detectedinformation preprocessing system 30, which corrects errors due todistortions and various aberrations in the focusing system, and extractsbright lines which satisfy a brightness or color condition required forextracting edges or obtaining three-dimensional distance information, ifnecessary. The image information thus obtained is sent to the detectedinformation storage system 40. When it is necessary to reduce falsecorresponding points by using four or more image focusing systems, theconfiguration of the focusing systems 11 to 13 and sensing surfaces 21to 23 is changed by the configuration change system 80, and the imageinformation obtained on the basis of the new configuration is also sentto the detected information storage system 40 through the preprocessingsystem 30. Next, the candidate-for-corresponding-point decision system50 extracts candidates for corresponding points from a multiplicity ofimage points stored in the storage system 40, by making use of a specialgeometrical relation, as mentioned in "SUMMARY OF THE INVENTION". Theextracted candidates are sent to the false-corresponding-point removingsystem 60, in which candidates corresponding to a false (namely,non-existing) object point are removed from the extracted candidates bymaking use of a brightness condition or others, as mentioned in "SUMMARYOF THE INVENTION". The remaining candidates are sent to thethree-dimensional distance information calculation system 70, to obtainthe three-dimensional distance information of the object system in amanner mentioned in "SUMMARY OF THE INVENTION".

According to the apparatus of FIG. 4, the configuration of the focusingsystems 11 to 13 and sensing surfaces 21 to 23 can be freely changed,and moreover, the information obtained on the basis of the newconfiguration is also stored. Thus, the apparatus of FIG. 4 can producethe same effect as an apparatus provided with four or more focusingsystems, and hence can greatly reduce the redundancy in extractingcandidates for corresponding points, no matter what structure the objectsystem may have.

Next, explanation will be made of a second embodiment of a method offorming a three-dimensional stereo vision in accordance with the presentinvention, by reference to FIG. 5.

The second embodiment is carried out by an apparatus which, as shown inFIG. 5, includes color filters 101 to 103 corresponding to three primarycolors, optical path length correction systems 112 and 113, mirrors 123,133, 121 and 131, a preprocessing system 200, an image focusing system300, a color image sensor 400, a horizontal addressing system 500, abrightness correcting circuit group 600, a brightness correction system700, an AND circuit group 800, a two-dimensional distance informationcalculation system 900, and a three-dimensional distance informationcalculation system 1000.

The color filters 101, 102 and 103 transmit red light, green light andblue light, respectively, and correspond to three primary color filtersmounted on the color image sensor 400. The color filters 101 to 103, theoptical path length correction systems 112 and 113, and the mirrors 123,133, 121 and 131 are disposed so that the light paths from the colorfilters 101 to 103 to the color imaee sensor 400 have the same opticalpath length, and respective optical axes of three light beams from thecolor filters 101 to 103 lie in the same horizontal plane and are spacedapart from one another at the light receiving surface of the color imagesensor 400 by an amount corresponding to the width (in a horizontaldirection) of each of the color filters mounted on the color imagesensor 400. As shwwn in FIG. 5, each of the mirrors 133 and 131 cantransmit light incident upon the back surface thereof.

Referring to FIG. 5, light which is emitted or reflected from an objectsystem and passes through an optical system which includes the colorfilters 101 to 103, the optical path length correction systems 112 and113 and the mirrors 123, 133, 121 and 131, is subjected to necessarybrightness modification processing and polarization processing in thepreprocessing system 200, and then is focused on the color image sensor400 by the image focusing system 300. The color image sensor 400 is ableto be addressed by horizontal direction addressing system 500 inrespective colors so that three pixels adjacent to one another andcorresponding to three colors in a horizontal direction which areprovided with the red, green and blue filters, can be separatelyaddressed by three address lines prepared independently for each colorand pixels arranged in a vertical direction and receiving light of thesame color can be simultaneously addressed. When the addressing for thecolor image sensor 400 is made in a horizontal direction and in apredetermined order, pixels corresponding to an address simultaneouslydeliver information corresponding to the color filters mounted on thesepixels, and the above information is sent to the brightness correctingcircuit group 600. The circuit group 600 includes three correctingcircuits for each of pixel layers arranged in a vertical direction. Theoutput of each pixel included in the color image sensor 400 is modifiedin accordance with that circuit constant of each correcting circuitwhich is specified by the brightness correction system 700, and thenthree color outputs from three pixels adjacent to one another in ahorizontal direction are applied to one of AND circuits included in theAND circuit group 800. The brightness correction system 700 gives eachcorrecting circuit of the brightness correcting circuit group 600 acircuit constant necessary for the correction of brightness balanceamong three primary colors, the brightness correction resulting from thecharacteristics of the object system, and the classification ofbrightness into a plurality of levels. Three color outputs having beensubjected to the brightness correction are applied to one AND circuit ofthe AND circuit group 800. When the three color outputs lie in apredetermined brightness range, the AND circuit delivers ON-information.When at least one of the three color outputs does not lie in thepredetermined brightness range, the AND circuit deliversOFF-information. The ON- or OFF-information thus obtained is applied toone two-dimensional distance information calculation circuit of thetwo-dimensional distance information calculation system 900. WhenON-information is applied to the above calculation circuit, the presenceof an object point is known, and the two-dimensional distanceinformation of the object point is obtained on the basis of the addressinformation from the horizontal addressing system 500 by the methodmentioned in "SUMMARY OF THE INVENTION". When the OFF-information isapplied to the two-dimensional distance information calculation circuit,the absence of an object point is indicated, and thus no processing iscarried out. The above two-dimensional distance information obtained ineach of pixel layers arranged in the vertical direction is sent to thethree-dimensional distance information calculation system 1000, to beused for obtaining the three-dimensional distance information of theobject system.

As can be seen from the above explanation, according to the secondembodiment, data from pixel layers arranged in the vertical directionare processed in parallel, and hence the processing time is greatlyshortened. Further, the whole processing can be carried out by hardware,and hence not only the productivity and reliability are improved butalso the processing speed is increased. Furthermore, only the lightreceiving surface of a single color image sensor is used as the imagesensing surface, and hence the apparatus for carrying out the secondembodiment is simple in structure and can be constructed at a low cost.

As has been explained in the foregoing, according to the presentinvention, a method of forming a three-dimensional stereo vision isprovided in which the processing time is short, and the fundamentalerror dependent upon the spatial distribution of object points can beavoided.

Further, the above mentioned according to the present invention has thefollowing advantages.

(1) Even when an object system has a special brightness pattern, precisedistance information can be formed by appropriately setting the numberand configuration of image focusing systems.

(2) When brightness information, color information and others are usedtogether with position information of each image point, unwanted data isremoved, and thus the processing can be performed more accurately and ata higher speed.

(3) Unlike a conventional pattern matching technique in which the wholeof an object system is processed at once, it is possible to specify aplurality of limited ranges of distance in the direction of depth, aplurality of limited ranges of horizontal azimuth angle and a pluralityof limited ranges of vertical azimuth angle in a desired order.Accordingly, only the three-dimensional distance information of thatportion of an object system which exists in the specified, limitedranges of the above factors, can be obtained, and further the presenceor absence of an object point in the above portion can be judged. Thus,efficient processing can be made.

(4) Unlike the conventional pattern matching technique in which thewhole of an object system is processed at once, it is possible todetermine corresponding points for an object point independently ofcorresponding points for another object point. Accordingly, even whenimage information contains a local error, only a portion of theprocessing for obtaining three-dimensional distance information isaffected by the error, and the accurate three-dimensional distanceinformation of object points which have no connection with the erroneousimage information, can be obtained.

(5) The processing speed can be greatly improved by using parallelprocessing at a portion of the whole processing.

(6) When preprocessing such as the extraction of edges included in anobject system, is carried out, the number of bright lines used isgreatly reduced, and the processing speed is further improved.

(7) The number of image focusing systems and the number of image sensingareas are required to be three or more in effect. Accordingly, anapparatus for carrying out the method according to the present inventionmay be constructed so that a single image focusing system and a singleimage sensing surface can be used in a time-divisional fashion.

In the foregoing explanation, the image of an object system is formed byan optical system. However, the present invention is not limited to theoptical system, but is applicable to various propagation systems such aselectromagnetic wave systems other than the optical system, an accousticsystem, an elastic wave system, and a particle beam system (for example,the electron beam of an electron microscope).

We claim:
 1. A method, of obtaining three-dimensional distanceinformation, comprising the steps of:forming images of an object systemon an image sensing surface by using at least three image focusingsystems so that at least three object images of the object system areformed on respective image sensing areas; correlating the image point ofan object point which are formed on said respective image sensing areasby said image focusing systems by using a relationship which existsbetween the geometrical positional relationships among said image pointsand the geometrical positional relationships among said image focusingsystems themselves and between said image focusing systems and saidimage sensing areas; and obtaining the three-dimensional distanceinformation of said object point by using information relating to thepositions of at least two of the corresponding image points on the imagesensing areas and information relating to the geometrical positionalrelationship between said two corresponding image points and the imagefocusing systems, wherein said correlating of the image points comprisesallocating the object images formed on said respective image sensingareas to respective imaginary planes which are superimposed with aspatial relationship corresponding to the spatial relationship of saidimage focusing systems, and detecting those image points on theimaginary planes which lie on the same straight line.
 2. A method ofobtaining three-dimensional distance information, comprising the stepsof:forming images of an object system on an image sensing surface byusing at least three image focusing systems so that at least threeobject images of the object system are formed on respective imagesensing areas; correlating the image points of an object point which areformed on said respective image sensing areas by said image focusingsystems by using a relationship which exists between the geometricalpositional relationships among said image points and the geometricalpositional relationships among said image focusing systems themselvesand between said image focusing systems and said image sensing areas;and obtaining the three-dimensional distance information of said objectpoint by using information relating to the positions of at least two ofthe corresponding image points on the image sensing areas andinformation relating to the geometrical positional relationship betweensaid two corresponding image points and the image focusing systems,wherein correlating of the image points comprises detecting image pointsin respective image sensing areas which satisfy the relationship thatthe ratio of the distance between first and second image points of thesame object point in respective first and second image sensing areas tothe distance between said second and a third image point of the sameobject point in respective second and third sensing areas is equal tothe ratio of the distance between first and second focusing system tothe distance between said second and a third focusing system.
 3. Amethod of obtaining three-dimensional distance information, comprisingthe steps of:forming images of an objection at least one image sensingsurface by using at least three image focusing systems so that at leastthree object images of the object are formed on said image sensingsurface; determining correspondence between image points of said imagesformed on said image sensing surface and object points on said object byusing a geometrical relationship in which a shape is formed by joiningsaid at least three image focusing systems, which shape moves towardsaid image sensing surface in parallel while maintaining a similar formso as to form a similar shape disposed between said shape and said imagesensing surface, each of the image points corresponding to the objectpoints existing on points of intersection of said image sensing surfaceand straight lines passing through vertices of said shape and thecorresponding vertices of said similar shape; and obtainingthree-dimensional distance information of said object points by usinginformation relating to the positions of at least two of said imagepoints formed on said image sensing surface corresponding to a certainobject point and information relating to the geometrical positionalrelationship between said two corresponding image points and said imagefocusing systems.
 4. A method according to claim 1, wherein each of saidimage sensing areas are formed on the same one-dimensional image sensingsurface.
 5. A method according to claim 1, wherein single image focusingsystem and a single image sensing surface are fixed successively atleast at three different positions and the image of said object systemis formed on said image sensing surface so as to produce at least threeobject images by using the formed image data on said image sensingsurface.
 6. A method according to claim 1, wherein a single imagefocusing system is fixed successively at different positions so that theimage of said object system can be formed on each of at least threeimage sensing surfaces.
 7. A method according to claim 1, wherein saidimage focusing systems are arranged on a straight line.
 8. A methodaccording to claim 1, wherein said image focusing systems are arrangedin a plane surface.
 9. A method according to claim 1, wherein said imagefocusing systems are arranged in a curved surface.
 10. A methodaccording to claim 1, wherein said image focusing systems are arrangedon a straight line at regular intervals.
 11. A method of obtainingthree-dimensional distance information, comprising the steps of:formingimages of an object system on an image sensing surface by using at leastthree image focusing systems so that at least three object images of theobject system are formed on respective image sensing areas; correlatingthe image points of an object point which are formed on said respectiveimage sensing areas by said image focusing systems by using arelationship which exists between the geometrical positionalrelationships among said image points and the geometrical positionalrelationships among said image focusing systems themselves and betweensaid image focusing systems and said image sensing areas; and obtainingthe three-dimensional distance information of said object point by usinginformation relating to the positions of at least two of thecorresponding image points on the image sensing areas and informationrelating to the geometrical positional relationship between said twocorresponding image points and the image focusing systems, wherein saidimage focusing systems are arranged in a plane surface so as to form aregular polygon.
 12. A method of obtaining three-dimensional distanceinformation, comprising the steps of:forming images of an object systemon an image sensing surface by using at least three image focusingsystems so that at least three object images of the object system areformed on respective image sensing areas; correlating the image pointsof an object point which are formed on said respective image sensingareas by said image focusing systems by using a relationship whichexists between the geometrical positional relationships among said imagepoints and the geometrical positional relationships among said imagefocusing systems themselves and between said image focusing systems andsaid image sensing areas; and obtaining the three-dimensional distanceinformation of said object point by using information relating to thepositions of at least two of the corresponding image points on the imagesensing areas and information relating to the geometrical positionalrelationship between said two corresponding image points and the imagefocusing systems, wherein said image focusing systems are arranged on astraight line at random.
 13. A method of obtaining three-dimensionaldistance information, comprising the steps of:forming images of anobject system on an image sensing surface by using at least three imagefocusing systems so that at least three object images of the objectsystem are formed on respective image sensing areas; correlating theimage points of an object point which are formed on said respectiveimage sensing areas by said image focusing systems by using arelationship which exists between the geometrical positionalrelationships among said image points and the geometrical positionalrelationships among said image focusing systems themselves and betweensaid image focusing systems and said image sensing areas; and obtainingthe three-dimensional distance information of said object point by usinginformation relating to the positions of at least two of thecorresponding image points on the image sensing areas and informationrelating to the geometrical positional relationship between said twocorresponding image points and the image focusing systems, wherein saidimage focusing systems are arranged in a plane surface so as to form aordinary polygon.
 14. A method of obtaining three-dimensional distanceinformation, comprising the steps of:forming images of an object systemon an image sensing surface by using at least three image focusingsystems so that at least three object images of the object system areformed on respective image sensing areas; correlating the image pointsof an object point which are formed on said respective image sensingareas by said image focusing systems by using a relationship whichexists between the geometrical positional relationships among said imagepoints and the geometrical positional relationships among said imagefocusing systems themselves and between said image focusing systems andsaid image sensing areas; and obtaining the three-dimensional distanceinformation of said object point by using information relating to thepositions of at least two of the corresponding image points on the imagesensing areas and information relating to the geometrical positionalrelationship between said two corresponding image points and the imagefocusing systems, wherein said image focusing systems are arranged in aplane surface parallel to said image sensing surfaces.
 15. A method ofobtaining three-dimensional distance information, comprising the stepsof:forming images of an object system on an image sensing surface byusing at least three image focusing systems so that at least threeobject images of the object system are formed on respective imagesensing areas; correlating the image points of an object point which areformed on said respective image sensing areas by said image focusingsystems by using a relationship which exists between the geometricalpositional relationships among said image points and the geometricalpositional relationships among said image focusing systems themselvesand between aaid image focusing systems and said image sensing areas;and obtaining the three-dimensional distance information of said objectpoint by using information relating to the positions of at least two ofthe corresponding image points on the image sensing areas andinformation relating to the geometrical positional relationship betweensaid two corresponding image points and the image focusing systems,wherein said image focusing systems are arranged in a plane surfacewhich is oblique with respect to said image sensing surfaces.
 16. Amethod of obtaining three-dimensional distance information, comprisingthe steps of:forming images of an object system on an image sensingsurface by using at least three image focusing systems so that at leastthree object images of the object system are formed on respective imagesensing areas; correlating the image points of an object point which areformed on said respective image sensing areas by said image focusingsystems by using a relationship which exists between the geometricalpositional relationships among said image points and the geometricalpositional relationships among said image focusing systems themselvesand between said image focusing systems and said image sensing areas;and obtaining the three-dimensional distance information of said objectpoint by using information relating to the positions of at least two ofthe corresponding image points on the image sensing areas andinformation relating to the geometrical positional relationship betweensaid two corresponding image points and the image focusing systems,wherein a plurality of operations corresponding to a range of verticalazimuth angles for correlating points are performed in parallel.
 17. Amethod of obtaining three-dimensional distance information, comprisingthe steps of:forming images of an object system on an image sensingsurface by using at least three image focusing systems so that at leastthree object images of the object system are formed on respective imagesensing areas; correlating the image points ff an object point which areformed on said respective image sensing areas by said image focusingsystems by using a relationship which exists between the geometricalpositional relationships among said image points and the geometricalpositional relationships among said image focusing systems themselvesand between said image focusing systems and said image sensing areas;and obtaining the three-dimensional distance information of said objectpoint by using information relating to the positions of at least two ofthe corresponding image points on the image sensing areas andinformation relating to the geometrical positional relationship betweensaid two corresponding image points and the image focusing systems,wherein a plurality of operations corresponding to a range of horizontalazimuth angles and a range of vertical azimuth angles for correlatingpoints are performed in parallel.
 18. A method of obtainingthree-dimensional distance information, comprising the steps of:formingimages of an object system on an image sensing surface by using at leastthree image focusing systems so that at least three object images of theobject system are formed on respective image sensing areas; correlatingthe image points of an object point which are formed on said respectiveimage sensing areas by said image focusing systems by using arelationship which exists between the geometrical positionalrelationships among said image points and the geometrical positionalrelationships among said image focusing systems themselves and betweensaid image focusing systems and said image sensing areas; and obtainingthe three-dimensional distance information of said object point by usinginformation relating to the positions of at least two of thecorresponding image points on the image sensing areas and informationrelating to the geometrical positional relationship between said twocorresponding image points and the image focusing systems, whereincorresponding points are determined in a limited range of verticalazimuth angles and in a limited range of horizontal azimuth angles. 19.A method of obtaining three-dimensional distance information, comprisingthe steps of:forming images of an object system on an image sensingsurface by using at least three image focusing systems so that at leastthree object images of the object system are formed on respective imagesensing areas; correlating the image points of an object point which areformed on said respective image sensing areas by said image focusingsystems by using a relationship which exists between the geometricalpositional relationships among said image points and the geometricalpositional relationships among said image focusing systems themselvesand between said image focusing systems and said image sensing areas;and obtaining the three-dimensional distance information of said objectpoint by using information relating to the positions of at least two ofthe corresponding image points on the image sensing areas andinformation relating to the geometrical positional relationship betweensaid two corresponding image points and the image focusing systems,wherein a plurality of limited ranges of vertical azimuth angles and aplurality of limited ranges of horizontal azimuth angles are specifiedin a desired order, and corresponding points are obtained in eachspecified, limited range of vertical azimuth angles and in eachspecified, limited range of horizontal azimuth angles.
 20. A method ofobtaining three-dimensional distance information, comprising the stepsof:forming images of an object system on an image sensing surface byusing at least three image focusing systems so that at least threeobject images of the object system are formed on respective imagesensing areas; correlating the image points of an object point which areformed on said respective image sensing areas by said image focusingsystems by using a relationship which exists between the geometricalpositional relationships among said image points and the geometricalpositional relationships among said image focusing systems themselvesand between said image focusing systems and said image sensing areas;and obtaining the three-dimensional distance information of said objectpoint by using information relating to the positions of at least two ofthe corresponding image points on the image sensing areas andinformation relating to the geometrical positional relationship betweensaid two corresponding image points and the image focusing systems,wherein image points corresponding to a non-existing object point areremoved on the basis of brightness information obtained on said imagesensing areas.
 21. An apparatus for obtaining three-dimensional distanceinformation, comprising:three color filtering means corresponding tothree primary colors and spaced apart from one another, for dividinglight from an object system into three light means; an image focusingsystem for forming the image of an object system by using said lightmeans of said object system received from said color filtering means;image sensing means formed of a color image sensor, said color imagesensor being provided with three kinds of color filters corresponding tosaid three color filtering means in such a manner that the three colorfilters are repeatedly mounted on pixels which form said color imagesensor, in a horizontal direction, to detect said three light beamsseparately, said pixels being scanned in a horizontal direction; opticaldeflection means for making the optical axes of said light beams fromsaid three color filtering means parallel to the axis of said imagefocusing system, for juxtaposing said optical axes of said 1ight beamsin a direction parallel to a horizontal pixel layer of said color imagesensor so that said optical axes deviate from one another, and forproviding the same optical path length between each of said three colorfiltering means and the image sensing surface of said image sensingmeans; horizontal addressing means for taking out information frompixels included in a horizontal pixel layer of said color image sensorin such a manner that a plurality of pixel groups each including threepixels which are provided with said three color filters, aresuccessively addressed in a predetermined order, and the ouputs of saidthree pixels of each pixel group are simultaneously taken out; detectionmeans for delivering ON-information when said three light beams fromsaid three color filtering means have a brightness value within apredetermined range, at those three pixels of a horizontal pixel layerwhich are addressed by said horizontal addressing means, and fordelivering OFF-information when at least one of said three light beamsdoes not have a brightness value within said predetermined range at oneof said three pixels; two-dimensional distance information calculationmeans for obtaining two-dimensional distance information which includesthe distance from said image sensing surface to said object system inthe direction of depth and at least one horizontal azimuth angle, fromthe horizontal address information on said color image sensor; andthree-dimensional distance information calculation means for obtainingthree-dimensional distance information which includes saidtwo-dimensional distance information of said object system and avertical azimuth angle, by using the vertical address information onsaid color image sensor.
 22. An apparatus according to claim 21, furthercomprising a brightness correction system for effecting brightnesscorrection for outputs of three pixels which are included in one pixelgroup and provided with said three color filters.
 23. An apparatusaccording to claim 21, wherein a contour extracting element is disposedin front of said image sensing surface.
 24. An apparatus according toclaim 21, wherein three polarizers and three analyzers for detectingsaid three light beams separately, perform substantially the samefunction as said three color filtering means and said three colorfilters, respectively.
 25. A method according to claim 3, wherein whensaid at least three image focusing systems are arranged on a line andsaid line is parallel with said image sensing surface, and whereindetermination of said correspondency is made by using the geometricalrelationship that a ratio between distances of adjacent image focusingsystems is the same as the ratio of distances of image pointscorresponding to the certain object point.
 26. A method according toclaim 3, wherein said at least three image focusing systems are arrangedon a common plane and said plane is perpendicular to said image sensingsurface.
 27. A method according to claim 3, wherein said at least threeimage focusing systems are arranged on a common plane parallel with saidimage sensing surface so that all of the image points corresponding to acertain object point exist on vertices of a figure on said image sensingsurface, said figure being similar to said shape formed on said plane.28. An apparatus for obtaining three-dimensional distance information,comprising:three color filtering means corresponding to three primarycolors and spaced apart from one another, for dividing light from anobject system into three light means; an image focusing system forforming the image of an object system by using said light means of saidobject system received from said color filtering means; image sensingmeans formed of a color image sensor, said color image sensor beingprovided with three kinds of color filters corresponding to said threecolor filtering means in such a manner that the three color filters arerepeatedly mounted on pixels which form said color image sensor, in ahorizontal direction, to detect said three light beams separately, saidpixels being scanned nn a horizontal direction; optical deflection meansfor making the optical axes of said light beams from said three colorfiltering means parallel to the axis of said image focusing system, forjuxtaposing said optical axes of said light beams in a directionparallel to a horizontal pixel layer of said color image sensor so thatsaid optical axes deviate from one another, and for providing the sameoptical path length between each of said three color filtering means andthe image sensing surface of said image sensing means; horizontaladdressing means for taking out information from pixels included in ahorizontal pixel layer of said color image sensor in such a manner thata plurality of pixel groups each including three pixels which areprovided with said three color filters, are successively addressed in apredetermined order, and the outputs of said three pixels of each pixelgroup are simultaneously taken out; detection means for deliveringON-information when said three light beams from said three colorfiltering means have a brightness value within a predetermined range, atthose three pixels of a horizontal pixel layer which are addressed bysaid horizontal addressing means, and for delivering OFF-informationwhen at least one of said three light beams does not have a brightnessvalue within said predetermined range at one of said three pixels;two-dimensional distance information calculation means for obtainingtwo-dimensional distance information which includes two horizontalazimuth angles, from the horizontal address information on said colorimage sensor; and three-dimensional distance information calculationmeans for obtaining three-dimensional distance information whichincludes said two-dimensional distance information of said object systemand a vertical azimuth angle, by using the vertical address informationon said color image sensor.